
On page 2, 3rd paragraph to page 4 : 



The approach to develop a sensory aid based on sensory substitution was guided by 
knowledge of the structure ands and functional organization of the visual system and 
by neurophysiological, psychophysical and behavioral studied on visual system 
recognition in a variety of animals. For example, honeybees use a photographic 
memory to store and recall patterns in a [[(]]pixel by pixel fashion[[)]]. However, 
they also distinguish patterns on the basis of global properties such as radial, circular, 
and bilateral symmetry. The honeybees honeybee's visual system possesses spatial 
filters for detecting various symmetries. Radial and circular filters have also been 
discovered in primate vision. This suggest that information about the presence or 
absence of different types of symmetry in a visual target may sometimes be useful in 
the differentiation and characterization of visual forms. In addition many insects that 
lack stereo vision use image-motion cues (differential movement parallax, changing 
size etc) to obtain information on depth. Image motion may also provide a powerful 
cue for segregating objects from their background, detecting imminent collisions, and 
for computing one's own motion. Likewise, the velocity flow field generated on the 
retina by different portions of a 3-D object as it approaches may produce sufficient 
information with which to reconstruct its form. These and other strategies such as 
visual velocity feedback for gaze and course control and the use of specialized 
scanning patterns for extracting information on specific features appear to be 
ubiquituous ubiquitous in vertebrate and invertebrate species where vision plays an 
important role in directing the animal's behaviour. Consequently any system of sense 
substitution that aims to compensate for the loss of vision, may well require the 
subject to implement a set of strategies that are functionally similar to those 
employed by the visual system in its analysis of the spatial and spatiotemporal 
features of the subjects subject's visual environment. 



On pa gg 10, 2nd paragraph: 

Returning to the specific example shown in Figure 1 , it can be seen that movement of 
the cursor 12 over the figure feature 10 will result in the playing of a musical 
sequence on which the ascending notes G, A, B, C of the second lowest octave and 
C, D, E, F of the second highest octave are played in succession. 

On page 12, 3rd paragraph: 

An important aspect of the present invention is the realisation that it is often 
advantageous to encode a subset of the full image into a musical sequence or 
sequences. Predetermined features can be extracted form from the image, and said 
predetermined features may be encoded into a musical sequence or sequences. 
Feature extraction or pattern recognition algorithms can be used for this purpose. 
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On page 12, 4th paragraph to page 14: 



For complex images it was necessary to extend the battery of search strategies 
available to our subjects to enable them to carry out a satisfactory exploration and 
analysis of the target. These included (1) a facility to mask or block out areas areas of 
the image to enable the subject to carry out a patch by patch exploration and analysis 
of the target and to selectively examine portions of the image. This was introduced in 
an attempt to simulate the tactile search pattern that a blind subject normally carries 
out when tactaully tactual ly exploring the shape of a solid object. (2) The second 
major change was to carry out a strategy of feature extraction to segment and produce 
a set of simplified representations of the image. In our original scheme, the entire 
image of the "object" was displayed on the computer screen and all pixels 
constituting the image were thereby primed to activate the release of the appropriate 
musical notes (that signalled their respective locations on the screen) when they were 
intercepted by the moving tracker bar. To be fully primed it was necessary and 
sufficient^,]] only that a pixel be occupied by part of the image on the screen. To 
facilitate the analysis of more complex visual shapes we introduced several additional 
conditions that needed to be fulfilled before a pixel is fully primed to activate the 
tracker bar. Effectively this is equivalent to having the full image of the "object" 
appearing on the monitor screen simultaneously by several different spatially 
distributed sets of feature detectors that each select only those pixels included in 
those parts of the image that display the selected feature to which a particular array of 
detectors is tuned. Thus one array of orientation selective detectors may select for 
priming any set of pixels that lie on well defined vertical lines in the image, while 
another array of feature detectors may do the same for horizontal lines etc. Further 
provision is made to allow the subject the choice (i) of selecting which feature (or 
combination of features) of the image are to be presented and (ii) which "voices" are 
to be excluded during a particular presentation. By screening the image with several 
sets of detectors which select for different features (e.g. horizontal, vertical and 
oblique line components), we can generate musical patterns associated with an image 
in which all but one of the features have been removed or an image that combines 
simultaneously^,]] a number of selected features (such as vertical and horizontal line 
compon e nts, or components) for sounding the whole figure i.e. by priming all the 
pixels and by arranging to scan and sound all the "voices" (segmented parts of the 
melody) simultaneously. Effectively this allows the subject to isolate and separately 
examine the contents of several segmented packages in which different features of 
the image have bee n se questered. 



On page 23, 3rd paragraph: 



The imaging means can comprise a video camera, although other means, such as 
CCD or photovoltaic detectors, might be employed. The encoding means performs 
the functions of analysing the image produced by the imaging means in a suitable 
manner, and encoding the analysed image into suitable musical sequences. The 
analysis step might comprise the division of the image, or portions of the image, into 
the desired number of pixels. It is of course highly desirable that the device is 
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portable, and thus a small, dedicated microprocessor might be used as encoding 
means. A small video camera can be used as part of a portable device: the video 
camera can be incorporated into a hand-held "wand, "wand." In both instances, 
scanning movements can be accomplished by the person via hand motion or 
automatically. The playing means can comprise an ear-piece worn by the person. 
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